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LEXINGTON 


MASSACHUSETTS 


ABSTRACT 


Although  little  known,  mathematical  morphology  offers  great  potential  in  the  areas 
of  image  enhancement,  feature  extraction,  and  object  recognition.  This  work  explores 
this  growing  field  through  a  survey  of  established  morphological  algorithms  and  the 
development  of  new  morphological  algorithms  for  range  image  analysis.  With  range 
imagery,  mathematical  morphology  is  used  for  noise  removal,  2-D  feature  extraction, 
3-D  feature  extraction,  and  3-D  corner  extraction. 
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Notation 


A,B,T1,T2,X 

subsets  of  Rn 

U 

union 

n 

intersection 

0 

empty  set 

Nc 

complement 

x,y,z 

vectors  in  Rr 

x€X 

x  is  £in  element  of  X 

A  C  X 

A  is  a  subset  of  X 

Ax 

translate  of  A  by  vector  x 

{x:P} 

set  of  points  satisfying  property  P 

dual  of  set  transformation  i.e., 

^*(^4)  =  [^(Ac)]c 

A  ©  B 

Minkowski  addition  of  sets  A  and  B,  i.e., 

A  ®  B  =  {x  +  y  :  x  €  A,  y  €  B}  =  UygB  Ay 

B 

symmetric  set  of  B,  i.e.,  B  =  {— x  :  x  €  B} 

A®  B 

dilation  of  A  by  B 

AQB 

Minkowski  difference  of  sets  A  and  B ,  i.e., 

A  ©  B  =  nv6B  Ay 

AQB 

erosion  of  A  by  B 

Ab 

opening  of  A  by  B,  i.e.,  Ab  =  (A  Q  B)  ®  B 

AB 

closing  of  A  by  B ,  i.e.,  AB  =  (A  ®  B)  Q  B 

A®T 

hit-or-miss  transformation  of  A  by  T  =  (T1,T2),  i.e., 

A  ©  (Tl,  T2)  =  (A  ©  Tl)  n  ( Ac  ©  T2) 
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/,  g  functions  defined  on  Rn 

Ros(f)  region  of  support  of  /  (region  over  which  /  is  defined) 
ip'  dual  of  function  transformation  ip,  i.e., 

</>*(/)  =  -V’(-Z) 

£/(/)  umbra  of  function  /,  i.e., 

U(f)  =  {(x,f)  :  f(x)  >t},x  6  Rn,t  6  R 
/  ©  g  Minkowski  addition  of  functions  /  and  g,  i.e., 

[/  ©  <7](z)  =  rnax[f{x  -  y)  +  g(y)],y  6  Ros(g) 
g(x)  symmetric  function  of  g,  i.e.,  g(x)  =  g(—x) 

f  ©  g  dilation  of  /  by  g 

U  reflected  umbra  U  =  {(x,f)  :  (x,—t)  6  U} 

f  ©  g  Minkowksi  difference  of  functions  /  and  g,  i.e., 

[/  0  =  rnin[f(x  -  y)  -  g(y)],y  €  Ros(g) 

fg  opening  of  /  by  g,  i.e.,  fg  =  (fQg)(Bg 

f 9  closing  of  /  by  g,  i.e.,  f9=(f(&g)Qg 

Mes(X)  Lebesgue  measure  of  set  X,  i.e.,  A les(X)  =  volume  of  X 
(X  €  R3),  Mes(X)  =  area  of  X  ( X  €  R2),  etc. 

Pr{P}  probability  of  event  P 
nB  nB  =  B  ®  B  ®  B . . .  n  times,  where  OB  =  0 

/  set  difference,  i.e.,  A/ B  =  A  fl  Bc 

Z  set  representing  a  measuring  window 

S(X )  skeleton  of  X,  i.e.,  {x:  Bx  is  a  disk  centered  at  x  and  contained 
in  X,  but  not  in  any  larger  disk  containing  Bx  and  included  in 

*} 

S6(X )  conditional  bisector  of  X,  i.e., 

S6(X )  =  U„>o(A  ©  nB)/[(X  e  (n  +  1  )B)  ®  SB) 
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XO  T 

thinning  of  X  by  T,  i.e.,  X  0  T  =  X/{X  ©  T) 

{Ti) 

sequence  of  hit-or-miss  SEs,  i.e.,  {T,}  =  {Ti,T2,...}  = 

{(IT,  t;'),  (r„  7?),...} 

x  o  m 

sequential  thinning  of  Ar  by  {T,},  i.e.,  AT  0  {T,}  =  (. . .  ((Ar  0 

Ti)  or*)...) 

{Ti  }» 

infinite  sequence  of  hit-or-miss  SEs 

conditional  dilation  of  x  by  B,  i.e.,  x  ©  B\XC  =  (x  ©  B)  H  Xc 

RB,(f),RD’(f) 

rolling  ball  transform  of  /  by  g,  i.e.,  RBg(f)  =  f  —  fg, 

RB'U)  =  f’-f 

K(h) 

global  covariance  of  a  set  as  a  function  of  vector  /i,  i.e.,  for  set 

A,  K(h )  =  Mes(X  n  X.h) 

cm 

local  covariance  of  a  set  as  a  function  of  vector  h,  i.e.,  for  set 

X,  C(h)  =  Pr{x,x  +  h  e  X) 
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Chapter  1 


Introduction 


Enabling  a  machine  to  see  is  very  difficult;  we  do  not  even  understand  how  we  ourselves 
see.  While  machines  cannot  see  very  well,  they  are  very  good  at  processing  numbers. 
Perhaps  this  is  why  modern  approaches  to  machine  vision  involve  much  numerical 
computation.  This  numerical  approach  has  one  major  drawback:  it  can  be  very  slow, 
so  slow  in  fact  that  real-time  operation  can  be  impossible.  Thus,  with  the  present 
approach  to  machine  vision,  one  can  see  a  need  for  fast  computer  algorithms  and 
architectures. 

With  this  need  for  fast  algorithms  and  architectures,  the  development  of  parallel 
image  processing  operators  becomes  attractive;  rather  than  process  a  large  numerical 
array  sequentially,  it  would  be  faster  to  distribute  the  computational  effort  across 
the  array,  computing  the  result  in  parallel.  Also,  the  development  of  general  image 
processing  operators  is  desirable;  the  use  of  a  few  simple  operators  for  many  different 
types  of  processing  would  greatly  simplify  hardware  design.  As  we  can  see,  general 
and  parallel  operators  are  very  appealing  for  machine  vision. 

Among  the  candidates  for  these  general/parallel  operators,  one  type  stands  out: 
the  operators  from  the  field  of  mathematical  morphology.  These  operators  show 
promise  as  general/parallel  image  operators,  as  well  as  being  founded  on  a  solid  math¬ 
ematical  basis. 
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Mathematical  morphology  (MM)  was  developed  in  the  mid-1960s  by  G.  Matheron 
and  J.  Serra  at  the  Paris  School  of  Mines  at  Fontainebleau,  France.  Their  intent  was 
to  build  a  solid  mathematical  foundation  for  studying  the  relationship  between  the 
geometric  and  milling  properties  of  ores.  Within  this  context  of  metallography  and 
petrography,  the  development  of  MM  algorithms  has  expanded.  However,  outside 
these  areas,  relatively  little  has  been  done  to  more  fully  explore  the  applications  of 
MM.  In  particular,  very  few  people  have  applied  the  techniques  of  MM  to  range  im¬ 
ages,  where  geometric  analysis  seems  especially  appropriate.  This  work  brings  to  light 
several  applications  of  MM  through  a  survey  of  established  morphological  algorithms. 
Also,  new  morphological  algorithms  for  range  image  analysis  are  developed. 

The  organization  of  this  thesis  is  as  follows.  Chapter  2  is  a  tutorial  on  MM. 
Chapter  3  is  a  survey  of  several  established  MM  algorithms.  Chapter  4  explores 
experiments  on  real  range  data  while  Chapter  5  covers  experiments  on  synthetic  range 
data.  The  conclusion  (Chapter  6)  is  followed  by  an  extensive  bibliography,  including 
comments  on  many  of  the  references. 
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Chapter  2 


Mathematical  Morphology 


This  chapter  presents  the  basic  operations  of  MM  and  then  some  of  the  special  prop¬ 
erties  relevant  to  their  implementation.  Most  of  the  formulas  are  taken  directly  out 
of  [26,34]  where  other  details  can  be  found. 


2.1  Basic  Operations 

The  basic  operations  are  presented  in  two  subsections.  The  first  corresponds  to  set 
MM  (used  for  analyzing  signals  which  can  be  thought  of  as  sets,  such  as  binary  images) 
and  the  second  to  function  MM  (used  for  analyzing  signals  which  are  thought  of  as 
functions,  such  as  gray-scale  images). 

2.1.1  Set  MM 

The  following  notation  is  used  throughout:  AtB,Tl,T2  subsets  of  Rn;  U  union;  (~1 
intersection;  0  empty  set;  [  ]c  complement;  x,y,z,  vectors  in  Rr;  {x  :  P}  set  of  points 
satisfying  property  P;  dual  of  set  transformation  i.e.,  ^'(^l)  =  [\1'(>1C)]C. 
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Minkowski  Addition  and  Dilation 


The  Minkowski  addition  ©  is  defined  as  A  ©  J9  =  {i  +  y:i£i,y6  B }.  One  can 
show  that 

A®B=  U  Av  =  |J  Bx,  (2.1) 

veB  xeA 

where  Ay  =  A  ©  {y}  is  the  translate  of  A  by  y.  The  set  {z  :  A  0  Bz  ^  0}  of  the  points 
z  such  that  A  hits  the  translate  Bz  is  called  the  dilation  of  A  by  B\  the  dilation  of  A 
by  B  is  also  equal  to  A  ©I?,  where  B  =  {— x  :  x  €  B}  is  the  symmetrical  set  of  B  with 
respect  to  some  origin  O.  For  these  and  all  other  MM  set  and  function  operations  the 
second  operand  will  be  referred  to  as  a  structuring  element  (SE)1. 

Minkowski  Subtraction  and  Erosion 

The  dual  of  ©  is  the  Minkowski  subtraction  ©,  i.e.,  A  ©  B  =  ( Ac  ©  B)c.  Using  (2.1) 
and  de  Morgan’s  theorem,  one  finds 

AQ  B  =  f)  Ay.  (2.2) 

v€B 

Note  that,  in  general,  AQ  B  ^  flxe>i  Bx.  The  dual  of  dilation  is  erosion.  The  erosion 
of  A  by  B  is  the  set  {z  :  Bz  C  A)  of  the  points  z  such  that  the  translate  Bz  is  included 
in  A ;  it  is  also  equal  to  AQ  B. 

Opening  and  Closing 

By  combining  the  previous  operations  we  obtain  the  opening  and  closing.  The  opening 
Ab,  and  the  closing  AB ,  of  A  by  B  are  defined  as 

Ab  =  (AQB)QB ;  AB  =  {AQB)QB.  (2.3) 

From  the  above  equation,  it  is  easy  to  show  that  Ab  =  ((AC)B)C,  thus,  these  operations 
are  duals  of  each  other.  Ab  can  be  interpreted  as  the  union  of  all  translates  of  B  which 
are  subsets  of  A.  AB  can  most  easily  be  interpreted  as  the  dual  of  the  opening. 

*A  SE  is  sometimes  referred  to  as  a  kernel. 
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Hit-or-miss  Transformation 


The  hit-or-miss  transformation  is  defined  as 

A  ®  (Tl,  T2)  =  (A  ©  Tl)  D  (Ac  ©  T2).  (2.4) 

A  point  z  is  in  A  (?)  (T1,T2)  if  and  only  if  Tl2  is  a  subset  of  A  and  T 2Z  is  a  subset 
of  Ac.  A  @  (Tl,  T2)  is  often  denoted  A  ®  T,  where  T  =  (Tl,  T2)  is  referred  to  as  a 
hit-or-miss  SE. 

Page  13  contains  a  summary  of  the  basic  operations  and  examples  of  dilation, 
erosion,  opening,  and  closing.  Example  1  illustrates  these  four  operations  in  the 
continuous  case  while  Example  2  shows  the  corresponding  results  computed  for  the 
discrete  case.  Note  that  the  dilation  and  the  closing  tend  to  fill  in  concave  corners 
and  indentations;  the  erosion  and  the  opening  tend  to  cut  off  convex  corners  and 
protrusions.  The  size  of  the  SE  determines  which  protrusions  will  be  cut  off  and  which 
indentations  will  be  filled  in.  Also,  note  that  opening  and  closing  generally  preserve 
the  size  of  the  input  set,  while  dilation  enlarges  the  input  and  erosion  shrinks  it. 

2.1.2  Function  MM 

The  notation  is:  /,  g  functions  defined  on  Rn;  x,y  vectors  in  Rn;  Ros(f)  region  of 
support  of  /  (region  over  which  /  is  defined);  xp'  dual  of  function  transformation  t/>, 
i.e.,  ip'(f)  =  -iK-/). 

The  umbra  U  of  a  function  /  is 

U(f)  =  {(x,  t )  :  /(x)  >  t)  X  €  Rn  t  €  R.  (2.5) 

The  correspondence  between  a  function  and  its  umbra  is  unique,  thus  we  can  define 
MM  function  operations  in  terms  of  MM  set  operations  on  the  umbras. 


12 


MINKOWSKI  ADDITION:  A©B  =  |»  +  b:»€A.  b€B|  EXAMPLE  2 


D 


< 

Cw 

x 

<  M 

u>  ^ 

>• 


II  © 

K  O 

<  < 

-%®  r 


n 


CD 

0 

< 

Z 

o 

p 

o 

< 

DC 

H 

CD 

D 

(/> 

2 

w 

$ 

o 

z 


CD 

01 

x 


it 

>GD 


X 

5 


< 

n 

Q 

% 

N 

CD 

c 

< 


CD 

0 

CD 

CD 

< 

0 

0 

II 

CD 

>0D 

< 

0 

0 

u 

*4 

< 

< 

CD 

n 

n 

m 

m 

N 

< 

< 

z  z  6  © 

o  o  z  z 

£  w  w  z 

5  °  o  uj 

_  QC  — >  a“ 

Q  UJ  O  O 


N 

>♦- 

0 

u 

< 

2  C 
O  P 

i  S 

o  " 

£  f? 

S  H. 

c  t 

*-  • 

w  < 

V) 


13 


Set  MM  tutorial 


Minkowski  Addition  and  Dilation 


The  Minkowski  addition  of  /  and  g  is  defined  through 

/  ffi  g  -  VU  ©  9)  =  U(f)  ©  U(g),  (2.6) 

where  the  arrow  indicates  the  correspondence  to  the  umbra  of  the  result.  Dilation  of 
/  by  g  is 

/  ©  g  -  U(f  0  g)  =  U(f)  ©  U(g ),  (2.7) 

where  g(x)  =  g{— x). 

Minkowski  Subtraction  and  Erosion 

The  Minkowski  subtraction  of  /  and  g  is  defined  through  /  ©  g  — ►  U(f  ©  g).  The 
right  hand  side  of  this  expression  cannot  be  written  as  U(f)  0  U(g)  because  U(g) 
extends  to  — oo  and,  according  to  Equation  2.2,  this  would  reduce  the  result  to  a 
point  at  — oo.  However,  since  0  and  ©  are,  by  definition,  dual  operations,  we  have 
/  ©  g  =  —[(—/)  ©  fit].  Before  continuing,  we  must  introduce  the  reflected  umbra 
U  =  {(x,t)  :  (a;,  —  t)  €  U}  and  two  of  its  properties 


U(f) 

=  [£/(-/)]•= 

(2.8) 

[WJ'ffit/ts)] 

which  can  be  easily  established.  Thus,2 

=  6(f)®  U(  g), 

(2.9) 

U(feg)  =y [-((-/)  ®! !)]  =[£((-/)  ®!l)]c 

=  r(-/Te  tf(5)]]‘  =  [</(-/)  ® 

=  [[W)]‘  ®  t>(s)]c  =u(f)ev(g), 

i.e., 

feg^UUeg)  =  U(f)eU(g).  (2.10) 

Erosion  of  /  by  g  is 

feg^U(fQg)  =  U(f)  ©  U(g).  (2.11) 

2This  derivation  is  somewhat  more  direct  than  that  given  in  [34]. 
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Opening  and  Closing 

The  opening  fg  and  closing  f9  of  /  by  g  tire  defined  as 

/i=(/ej)e  g;  /'«(/©#)©*.  (2.12) 

Relations  (2.6)  and  (2.10)  can  be  expressed  algebraically  as 

[/  ©  slOO  =  max[f(x  -  y)  +  flf(y)]  y  e  Ros(g)  (2.13) 

[/  ©  5](*)  =  min[f(x  -  y)  -  g(y)}  y  G  Ros(g),  (2.14) 

where,  by  convention,  /  =  — oo  outside  Ros(f).  Dilation  and  erosion  are  given  by 

[/  ©  ff](*)  =  max[f( x  +  y)  +  y(y)]  y  G  Ros(g)  (2.15) 

[/©5](a;)  =  min[f(x  +  y)  -  g(y)]  y  £  Ros(g).  (2.16) 


This  leads  to  relatively  simple  algorithms  for  computation  of  the  function  MM  oper¬ 
ations.  Note  that  (2.13),  (2.14),  (2.15),  and  (2.16)  sure  the  morphological  equivalents 
of  the  standard  convolution  and  correlation. 

Page  16  contains  a  summary  of  the  basic  operations  and  examples  of  dilation, 
erosion,  closing,  and  opening.  Example  1  illustrates  these  four  operations  in  the 
continuous,  1-D  case  while  Example  2  shows  the  corresponding  results  computed  for 
the  discrete,  2-D  case.  As  with  set  MM,  the  dilation  and  the  closing  tend  to  fill  in 
concave  corners  and  indentations;  the  erosion  and  the  opening  tend  to  cut  off  convex 
corners  and  protrusions.  The  size  of  the  SE  determines  which  protrusions  will  be 
cut  off  and  which  indentations  will  be  filled  in.  Also,  note  that  opening  and  closing 
generally  preserve  the  size  (height  and  region  of  support)  of  the  input  function,  while 
dilation  enlarges  the  input  and  erosion  shrinks  it. 
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Function  MM  tutorial. 


2.2  Properties  Relevant  to  Implementation 


This  section  covers  three  of  the  important  properties  of  Minkowski  addition  and  sub¬ 
traction  related  to  the  implementation  of  the  basic  MM  operations. 

The  distributivity  property  is 

A  ©  (-Si  U  B2^  =  (j4  ©  B\ )  U  (^4  ©  B2)  (2.17) 

A  ©  (B\  U  B2 )  =  (A  ©  B\ )  D  ( A  ©  Bj')-  (2.18) 

This  property  implies  that  one  can  dilate  or  erode  A  by  taking  ( B\  \JB2)  piece  by  piece 
and  then  combining  the  intermediate  results  by  union  or  intersection.  This  property 


also  applies  to  functions,  i.e., 

f  ®max(gi,g2)  =  max(f  ©  gu  f  ©  g2)  (2.19) 

f  Qmax(gi,g2)  =  min(f  ©  gu  f  ©  g2).  (2.20) 

The  iterativity  property  is 

A  ©  (Bj  ©  B2)  =  (A®Bi)®B2  (2.21) 

A  ©  ( B\  ©  B2 )  =  (A  ©  B\ )  ©  B2.  (2.22) 


This  property,  which  applies  to  functions  as  well,  implies  that  one  can  decompose 
a  SE  into  a  Minkowski  sum  of  simpler  SEs  and  then  iterate  the  dilation  or  erosion 
to  get  the  intended  result.  This  decomposition  property  is  particularly  important 
to  the  implementation  of  morphological  operators  on  cellular  logic  arrays  such  as  the 
GLOPR,  CLIP,  or  Cytocomputer  (see  [53]  for  a  comparison  of  these  computers)  which 
can  access  only  a  local  neighborhood  of  pixels,  such  as  a  3  x  3  window.  Zhuang  and 
Haralick  [46]  have  shown  how  to  optimally  decompose  set  MM  SEs  through  a  tree 
search  of  possible  decompositions. 

The  property  of  Minkowski  addition  and  subtraction  which  leads  to  the  fastest 
implementation,  however,  comes  from  the  following  expressions  for  these  operations. 
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Recall  that  for  sets 


A  ©  B  —  Ay 
yCB 

A  e  B  =  n  Ay 

veB 

and  for  functions 


[/©^K*)  =  max[f{x-y)  +  g(y)]  y  6  Ros(g) 

[/ ©$](*)  =  min[f(x-y)-g(y)}  y  e  Ros(g). 


This  implies  that  in  the  set  MM  case,  the  morphological  results  can  be  computed 
as  logical  combinations  of  shifted  versions  of  the  input  image.  In  the  function  MM 
case,  the  morphological  results  can  be  computed  by  shifting  an  input  image,  adding 
or  subtracting  a  SE  value  from  the  shifted  image,  and  then  taking  the  max  or  min  of 
this  and  the  accumulated  result. 


2.3  Implementation 

For  this  work,  both  erosion  and  dilation3  have  been  implemented  (for  sets  and  func¬ 
tions)  as  the  basic  MM  operations;  all  others  such  as  Minkowski  subtraction,  Minkowski 
addition,  opening,  closing,  etc.,  are  implemented  as  combinations  of  these  basic  op¬ 
erations.  All  operations  have  been  programmed  in  LISP  and  C  under  the  SKETCH 
[69]  image  understanding  operating  system  developed  at  the  MIT  Lincoln  Laboratory. 
This  tool  was  used  for  all  experiments  described  in  Chapters  4  and  5. 


3Dilation  can  be  implemented  as  the  complement  of  the  erosion  of  the  complement  (duality). 
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Chapter  3 


Previous  Work 


This  chapter  discusses  MM  algorithms  developed  through  traditional  applications 
of  MM.  These  applications  include  texture  analysis,  defect  detection,  image  coding, 
and  biological  cell  analysis.  Throughout  the  next  several  sections  the  discussion  will 
be  geared  toward  the  development  of  an  intuitive  understanding  of  MM  algorithms, 
rather  than  the  more  rigorous  presentation  found  in  [26,34].  The  first  sections  discuss 
techniques  used  in  the  processing  of  range  imagery  in  Chapters  4  and  5.  The  last  few 
sections  (morphological  size  and  shape  description,  edge  detection  and  perimeter  es¬ 
timation,  and  covariance)  explore  several  other  useful  algorithms  which  are,  however, 
not  used  in  later  chapters. 

The  notation  is:  Mes(X)  Lebesgue  measure  of  set  X,  i.e.,  Mes(X)  =  volume  of 
X  (X  6  R3),  Mes(X)  =  area  of  X  (X  6  R2),  etc;  Pr{P }  probability  of  event  P\ 
nB  =  B ©  B ®B  . . .  n  times1  ;  /  set  difference,  i.e.,  A/B  =  AC\BC\  Z  set  representing 
a  measuring  window. 

1OB  =  0,1B  =  B,2B  =  B®B  ... 
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Figure  3. 1.  Skeletons. 


3.1  Skeleton,  Thinning,  and  Conditional  Bisector 

3.1.1  Skeleton 

A  point  x  is  part  of  the  skeleton  of  the  set  X  if  the  following  condition  is  satisfied: 

•  If  Bx  is  a  disk  centered  at  x  and  contained  in  X ,  there  is  no  larger  disk  containing 
Bx  and  included  in  X. 

The  set  Bx  is  called  a  maximum  disk  and  it  is  the  center  of  each  of  these  maximum 
disks  which  make  up  the  skeleton.  Examples  of  skeletons  sire  shown  in  Figure  3.1. 
The  skeleton  can  be  computed  in  the  following  way.  If  B  is  the  smallest  disk  on  the 
image  grid  (Fig.  3.2),  the  skeleton2  S(X)  is  given  ([34]  p.  389)  by 

S( X)  =  (J  sn( X)  n  integer  (3-1) 

n>0 

sn(X)  =  (XenB)/(XQnB)B.  (3.2) 

The  original  set  X  can  be  reconstructed  from  the  skeleton  subsets  sn(X )  as  follows: 

2The  skeleton  is  virtually  identical  to  the  medial  axis  transform  [60,34].  See  [34]  for  a  more  thorough 
treatment. 
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+ 

Figure  3.2.  Smallest  disk  on  the  square  grid . 

X  =  U(sn(*)©n£).  (3.3) 

n 

The  capability  of  representing  binary  images  as  skeletons  (along  with  the  associated 
disk  size  for  each  skeleton  subset)  provides  for  efficient  coding  of  binary  images  [25]. 

3.1.2  Thinning 

Another  transformation  which  is  similar  to  the  skeleton  is  the  homotopic  sequential 
thinning. 

Thinning ,  denoted  0)  *s  811  application  of  the  hit-or-miss  transformation  and  is 
defined  for  T  =  (T',T")  ([34]  p.  390)  as 

XOT  =  X/(X®T).  (3.4) 

An  example  is  shown  in  Figure  3.3  where  T1  appears  in  black  and  T"  in  white  within 
the  3x3  neighborhood.  The  origins  of  both  T'  and  T"  are  at  the  center  of  the  3  x 
3  neighborhood  and  points  of  the  3x3  neighborhood  which  belong  to  neither  T'  nor 
T"  are  marked  with  dots.  This  pictorial  representation  (of  hit-or-miss  SEs  used  for 
thinning)  will  be  used  throughout  the  rest  of  this  work. 

Sequential  thinning  involves  thinning  by  a  sequence 

{TJ  =  {T„  7i, . . .}  =  {(t;,  t"),  (t;,  T"), . . .} 

of  hit-or-miss  SEs.  The  sequential  thinning  of  X  by  the  sequence  {Tl}  is  defined  as 

A'  O  {!}}  =  (.. .  ((A  or,)  O  T,). . .).  (3.5) 

If  the  sequence  {T,}  is  infinite  (denoted  {T,}oo),  the  input  X  is  typically  thinned  until 
there  is  no  change  from  one  iteration  to  the  next. 
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T  =  (T.T") 


Figure  3.3.  Thinning. 


Homotopic  thinning  is  a  special  case  of  thinning.  To  describe  what  is  meant 
by  homotopic,  first  consider  the  homotopy  tree  associated  with  the  set  X  shown  in 
Figure  3.4  (shaded  region).  The  root  of  this  tree  corresponds  to  the  background 
Xq  (i.e.,  the  infinite  connected  component  of  Xc );  the  first  branches  correspond  to 
the  connected  components,  Xi  and  X[ ,  of  X  adjacent  to  Xo;  the  second  branches 
correspond  to  the  holes  of  Xi,  adjacent  to  X1(  etc.  A  transformation  is  said  to  be 
homotopic  if  it  does  not  modify  the  homotopy  tree  of  the  input  set  X.  Thus,  there  is 
a  one-to-one  correspondence  between  connected  components  of  X  and  the  connected 
components  of  the  homotopic  thinning  of  A'. 

Homotopic  sequential  thinning  is  sequential  thinning  with  a  homotopy-preserving 
SE  sequence.  An  example  of  such  a  sequence  is  the  set  of  consecutive  90-degree 
rotations  of  the  Levialdi  [63  letters  L  and  L’  shown  in  Figure  3.5.  Sequential  thinning 
of  the  set  A  by  the  sequence  {T}OT  (Fig.  3.5)  results  in  a  thin  line  drawing  of  X.  The 
result  is  a  kind  of  “skeleton”,  although  in  this  case  the  homotopy  of  X  is  preserved. 
An  example  is  shown  in  Figure  3.6. 
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Figure  3.4.  A  set  and  its  homotopy  tree. 


bb  an  be  is  m  as 

lDE  kAri  l«l  1  I  I  i  i*i  i m 


Figure  3.5.  {z.}  ao. 


X  X  O  (LU 

Figure  3.6.  Homotopic  sequential  thinning 
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The  homotopic  sequential  thinning,  in  combination  with  the  hit-or-miss  trans¬ 
formation,  can  be  useful  in  detecting  certain  types  of  defects  [28,20].  For  example, 
Mandeville  [20]  has  used  the  homotopic  sequential  thinning  of  printed  circuit  board 
images  to  detect  pixel  patterns  which  indicate  circuit  defects. 

3.1.3  Conditional  Bisector 

The  conditional  bisector  is  a  subset  of  the  skeleton  and  is  given  by  the  following 
formula  ([34]  p.  390,  [31]): 

S\X)  =  \J(XenB)/[(Xe(n  +  l)B)@9B) 

n>  0 

=  |J  (*©"*)/[(*  ©»»S)b®(0-1)S],  (3.6) 

n>0 

where  9  is  an  integer  and  B  is  the  smallest  disk  of  the  grid.  Note  that  the  denominator 
of  the  second  expression  is  derived  from  the  first  as  follows: 

(X  ©  (n  +  1)B)  ®  6B  =  (X  ©  (nB  ©  B))  ©  (B  ©  (6  -  1)B) 

=  (((X  e  nB)  ©  B)  0  B)  ©  (9  -  1)B 
=  (XenB)B®(9-l)B. 

When  9  is  one,  the  conditional  bisector  is  just  the  skeleton. 

One  interpretation  of  the  conditional  bisector  is  the  part  of  the  set  ( X  ©  nB)  not 
reached  by  the  set  (X  ©  (n  +  1  )B)  ©  9B.  If  (Ar  ©  (n  +  1  )B)  =  0,  then  the  ultimate 
erosion  has  been  found.  The  ultimate  erosion  of  X  is  the  erosion  by  the  largest  disk 
nB  such  that  X  ©  nB  ^  0.  Meyer  [31,28]  has  used  this  property  of  the  conditional 
bisector  (along  with  the  thinning  and  the  hit-or-miss  transformation)  for  detecting 
overlapping  or  non-convex  biological  cells.  A  pictorial  representation  of  the  algorithm 
is  shown  in  Figure  3.7. 
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qp 


INDICATION  OF 
OVERLAPPING 
OR  NON-CONVEX  CELLS 


ORIGINAL  SETS  SET  DIFFERENCE  OF  THINNINGS 

ORIGINAL  SETS  AND 
CONDITIONAL  BISECTORS 

Figure  3.7 .  Application  of  the  conditional  bisector. 

3.2  Region  Painting,  Labelling,  and  Extraction 

Given  the  boundary  X  of  a  region  (e.g.  a  one-pixel  wide  closed  curve)  and  a  single 
point  x  of  the  interior,  we  can  use  MM  to  fill  in  the  region  [22].  The  first  step  in 
“painting”  the  region  consists  of  dilating  the  interior  point  x  by  the  smallest  disk  B 
of  the  grid.  Then  this  result  is  intersected  with  the  complement,  Xc ,  of  the  boundary 
X.  This  operation  of  restricting  the  dilation  to  a  certain  region  is  called  conditional 
dilation  and  is  denoted  x  ©  B\XC  ([34]  p.  393)3  .  That  is, 

x®B-,Xc  =  (x®B)DXc.  (3.7) 

The  next  step  toward  painting  the  region  is  to  use  the  results  of  the  first  conditioned 
dilation  as  input  to  the  next  conditioned  dilation.  This  process  is  iterated  until  the 
region  is  filled. 

Conditional  dilation  by  a  sequence  can  also  be  used  for  region  labelling  and  ex¬ 
traction  ([34]  p.  405).  If  an  input  set  X  consists  of  several  disjoint  regions  X,- ,  i.e., 
X  =  (Jt- J\Tt,  we  can  extract  and  label  the  Xi  using  the  following  algorithm: 

3The  concept  of  a  conditional  operation  can  also  be  extended  to  erosion,  thinning,  closing,  etc. 
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Figure  3.8.  Rolling  ball  transform  (l-D) 

1.  Scan  the  input  image  and  stop  at  the  first  point  x,-  6  A  (i  =  1  initially).  X,  =  x,-. 

2.  A,  =  A,  ©£;A. 

3.  If  step  2  produced  no  change  in  A,-,  go  to  4,  else  go  to  2.  This  fills  in  the  A,. 

4.  A  =  A/A,,  i  =  i  +  1,  go  to  1.  This  subtracts  out  the  previously  labelled  A,-. 

3.3  Rolling  Ball  Transform 

The  rolling  ball  transform,  [40]  is  applied  primarily  to  functions  although  the  same  idea 
can  be  extended  to  sets.  If  g  is  a  hemispherical  function,  the  opening  fg  is  a  function 
defining  the  places  where  the  hemisphere  fits  beneath  the  surface  of  /.  The  rolling  ball 
transform  RBg(f )  =  /  —  fg  then  gives  all  the  regions  where  the  hemisphere  g  does  not 
fit  beneath  the  surface  of  /.  This  is  illustrated  in  Figure  3.8  for  a  l-D  function.  If  the 
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hemisphere  g  is  inverted  and  moved  above  the  function  /,  the  rolling  ball  transform 
is  denoted  RB9(f)  and  is  equal  to  / 9  —  f.A 

One  application  of  the  rolling  ball  transform  is  background  normalization.  Stern¬ 
berg  [40]  has  used  it  for  this  purpose  on  images  of  human  cells. 


3.4  Morphological  Size  and  Shape  Description 

The  material  in  this  and  the  next  sections  is  not  exploited  in  Chapters  4  and  5  but 
is  included  because  of  its  importance  in  the  classical  applications  of  MM.  Two  MM 
techniques  which  extract  size  and  shape  information  axe  linear  erosion  and  the  opening 
with  a  disk  (or  sphere  in  3-D). 

The  linear  erosion  ([34]  p.  323)  involves  eroding  the  input  set  X  by  a  segment  of 
length  /  in  direction  a,  !?(/,  a).  One  can  obtain  a  size  distribution  for  a  fixed  direction 
ao  by 

A fes(X  Q  B(l,a0))  where /  = /o, /i, .. .  (3-8) 

The  size  and  shape  distribution  can  also  be  measured  using  the  opening  of  the 
input  by  nB,  a  disk  of  radius  n.  By  varying  the  radius  n  and  talcing  the  difference 
between  consecutive  openings,  the  size  and  shape  distribution  can  be  measured,  i.e., 

Mes(XnB )  —  Afes(AL(n+1)B).  (3.9) 

Examples  of  size  and  shape  distributions  (differences  between  consecutive  openings) 
axe  shown  in  Figure  3.9.  Maxagos  [24]  has  used  the  differences  between  consecutive 
openings  and  the  dual  notion  of  differences  between  consecutive  closings  in  defining 
the  pattern  spectrum  of  a  binary  image. 

Both  lineax  erosion  and  opening  by  disks  have  probabilistic  versions  ([34]  p.  360). 
The  primary  difference  is  that  the  measurements  described  above  are  divided  by  the 

4Since  there  is  no  standard  notation  for  the  rolling  ball  transform,  RBt(f)  and  RB,(f)  were 
arbitrarily  selected. 
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Figure  3.9.  Size  and  shape  distributions  as  a  function  of  disk  size  n. 


total  number  of  occurences  (in  the  measuring  window)  of  the  particular  segment  or 
disk.  For  example,  for  linear  erosion  measurements  within  the  window  Z,  Equation  3.8 
becomes 

Mea{XeB{h<*))  ,,  1Qx 

Mes(Z  ©  B(/,<*))'  1  ’  ] 

3.5  Edge  Detection  and  Perimeter  Estimation 

Minkowski  addition  and  subtraction  can  be  used  to  locate  edges  in  an  image.  For  a 
sufficiently  small  disk,  B,  of  radius  r,  the  edges  of  a  set  X  axe  given  by 

(X  ©  B)/(X  ©  B).  (3.11) 

The  perimeter,  U(X),  of  X  can  also  be  computed  using  Minkowski  addition  and 
is  approximately  equal  to 

Mes((X  ®  B)/X).  (3.12) 

To  see  this,  remember  that  the  Minkowski  addition  of  a  small  disk  and  the  set  X  has 
the  effect  of  “growing”  a  new  layer  of  “skin”  on  X .  Talcing  the  difference  between  this 
new  set  and  the  original  leaves  only  the  “skin”,  the  measure  of  which  is  approximately 
equal  to  the  perimeter. 

Equations  3.11  and  3.12  can  be  extended  to  functions  for  surface  area  estimation 
and  gray-scale  edge  detection  by  using  a  hemispherical  function  as  the  SE. 
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3.6  Covariance 


Most  of  the  MM  techniques  available  in  the  literature  have  come  about  through  work 
on  texture  analysis.  One  of  the  most  useful  techniques  for  this  purpose  is  what  Serra 
([34]  p.  271)  calls  the  covariance.  The  covariance  is  useful  for 

•  extracting  periodicities  from  what  may  seem  to  be  a  uniform  texture 

•  analyzing  the  superposition  of  scales  in  an  image 

•  detecting  anisotropies  at  a  given  scale 

The  following  subsections  discuss  both  the  global  covariance  and  the  point  covariance 
(which  is  a  probabilistic  version  of  the  global  covariance),  and  sin  application  of  each. 

3.6.1  Global  Covariance 

If  B  is  the  SE  consisting  of  two  points  {0,  h)  ( h  is  a  vector  with  magnitude  \h\  and 
direction  a),  the  global  covariance  K(h),  defined  as  Mes{x  +  X},  is  given 

by 

K(h)  =  Mes(X  ©  B)  =  Mes( X  fl  X.h).  (3.13) 

If  k(x)  is  the  characteristic  function  associated  with  the  set  X,  the  covariance  can 
also  be  viewed  as 

K{h)=  [  k(x)k(x  +  h)dx.  (3.14) 

J  R" 

The  following  are  several  properties  of  K(h):  K( 0)  =  Mes(X);  K(h )  =  K(—h)\ 
K(0)  >  A'(/i).  An  example  of  global  covariance  computation  can  be  seen  in  Fig¬ 
ure  3.10. 

In  most  practical  situations  AT'(O)  =  lim|^|_o  — (AT(0)  —  K{h))/\h\  exists  and  it  is 
equal  to  the  total  projection  area  (X  6  R3)  or  total  projection  diameter  ( X  6  R2) 
of  X  in  direction  a.  An  example  of  the  total  projection  diameter  is  illustrated  in 
Figure  3.11  where  it  is  equal  to 
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K(h) 


Figure  3.10.  Global  covariance  (adapted  from  [34]). 


a 


Figure  3.11. 


Total  diameter  of  X  ( ^  4.)- 
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3.6.2  Point  Covariance 


If  one  can  observe  the  set  X  only  within  a  limited  window  Z,  the  point  covariance 
C(h),  defined  as  Pr{x,x  +  h  £  X},  may  be  a  more  suitable  measure  than  the  global 
covariance  Mts{x  :  x,x  -f  h  £  X}.  A  good  estimate  ([34]  p.  281)  of  C(h )  is  C"*(/i) 
given  by 


C\h)  = 


Mes((X  n  Z)eB) 


(3.15) 


Mes{Z  ©  B) 

The  denominator,  Mts{Z  ©  B ),  is  the  total  possible  number  of  occurences  of  B  = 
{0,  h}  within  the  measuring  window  Z.  The  properties  of  C(h )  axe:  C(0)  =  Pr{x  € 
X };  C(h)  =  C(-h);  C(0)  >  C(h). 


3.6.3  Applications 

The  global  covariance  has  been  used  to  study  the  spatial  distribution  of  cells  in  the 
embryonic  ovary  of  a  rat  [10].  In  this  application,  covariance  was  used  to  determine 
if  the  cells  were  uniformly  distributed  within  the  ovary.  Once  it  was  determined  that 
they  were  clustered,  the  covariance  was  used  at  a  smaller  scale  to  measure  the  average 
distance  between  cell  centers  within  each  cluster. 

The  point  covariance  has  been  used  to  study  the  relation  of  wood  structure  to 
anisotropies  of  shrinkage  during  drying  [36].  In  this  application,  tree  cross-sections 
were  analyzed  radially  and  tangentially,  at  several  scales,  to  find  correlations  between 
the  wood  structure  and  its  shrinkage  patterns. 
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Chapter  4 


Experiments  with  Real  Range 
Imagery 


This  chapter  begins  with  a  discussion  of  the  concept  of  a  range  image.  Following  this, 
the  real  range  data  is  described  and  related  MM  algorithms  are  presented. 


4.1  Range  Images 

The  value  associated  with  a  point  in  a  range  image  is  proportional  to  the  distance 
between  the  range  sensor  and  the  corresponding  point  in  the  3-D  physical  world. 
A  range  image  can  be  interpreted  as  a  2-D  function  having  a  3-D  graph  which  is 
actually  an  inverted  version  of  the  physical  surface  being  imaged  (Fig.  4.1).  Thus, 
the  operation  of  rolling  a  ball  above  the  surface  of  the  object  corresponds  to  moving 
a  hemispherical  function  beneath  the  image  function  (Fig.  4.1).  This  inversion  of 
surfaces  must  be  kept  in  mind  when  thinking  intuitively  about  the  notions  of  closing, 
opening,  etc. 
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Figure  4.1.  Range  image  generation. 
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4.2  Real  Data 


The  real  range  images  are  obtained  from  a  laser  radar.  For  each  pixel  in  a  range 
image,  a  radar  pulse  is  sent  out  into  the  world  and  the  time  between  the  transmission 
of  the  pulse  and  the  reception  of  the  reflected  pulse  is  measured.  If  no  reflection  is 
received  within  a  certain  time  window,  the  pixel  is  called  a  dropout.  For  other  pixels, 
the  value  recorded  is  directly  proportional  to  the  distance  to  the  reflection  point  in 
the  world. 

Since  the  data  is  quantized  to  8  bits,  each  pixel  value  must  be  within  the  range 
0-255.  With  this  data  set,  the  values  0,  1,2,  and  3  turn  out  to  correspond  to  dropouts. 
Note  that  reflected  pulses  which  are  received  too  early  axe  recorded  as  dropouts,  as 
are  reflections  which  are  received  too  late  (or  not  at  all). 

The  radar  used  in  this  work  scans  a  square  section  of  the  world,  however,  the 
output  of  the  scanner  is  a  rectangular  image  (128  x  60  pixels).  This  occurs  because 
the  vertical  angular  sampling  interval  of  the  scanner  is  twice  that  of  the  horizontal 
angular  sampling  interval.  Because  of  this  disparity,  the  original  and  processed  images 
have  been  zoomed  vertically  by  a  factor  of  2  for  display  purposes,  although  processing 
is  performed  on  the  original  128  X  60  pixel  images.  An  example  of  an  unzoomed  real 
range  image  is  shown  on  page  35. 


4.3  Experiments 

In  this  section,  the  problems  of  noise  removal  and  2-D  feature  extraction  are  addressed. 

4.3.1  Noise  Removal 

There  are  two  types  of  noise  in  the  real  range  images:  dropouts  and  outliers.  As 
indicated  above,  dropouts  have  one  of  the  values  0,  1,  2,  or  3,  thus,  they  appear  as 
dark  spots.  Outlier  pixels  have  values  far  from  those  of  neighboring  pixels  and 
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appear  as  both  bright  and  dark  spots.  Since  there  are  two  different  types  of  noise,  two 
different  methods  of  noise  removal  should  be  employed.  In  fact,  the  situation  is  even 
more  complex  since  both  dropouts  and  outliers  can  appear  as  dark  spots.  Therefore, 
blind  removal  of  dark  spots  cannot  be  attempted  before  the  dropouts  are  filled  in. 
Thus,  the  proposed  cleaning  procedure  is: 

1.  Remove  outliers  which  appear  as  bright  spots. 

2.  Fill  in  values  for  dropouts. 

3.  Remove  outliers  which  appear  as  dark  spots. 

To  perform  these  processing  steps,  only  function  MM  and  point  operations  are  used. 
The  results  and  the  SEs  which  were  used  are  shown  on  page  37.  Each  of  the  SEs  is  a 
flat  2-D  function  with  value  zero  inside  the  region  of  support  and  origin  marked  0. 

To  remove  the  outliers  which  appear  as  bright  spots,  we  take  the  pointwise  min¬ 
imum  of  the  original,  say  i  (p.  37-1),  and  its  dilation  t  ®  a  (p.  37-2),  i.e.,  t  = 
mm(i,(z  ©  a))  (p.  37-3).  If  one  looks  at  the  formula  for  dilation  (Eq.  2.15),  the 
value  of  t  at  a  point  x,  f(x),  can  be  expanded  to 

<(x)  =  [mzn(z,(z  ©a))](x)  =  [mm(t,(t  ©a))](x) 

=  min(i(i),mai(i(i  —  y)  -f  0))  y  €  Ros(a). 

By  noting  that  Ros(a)  is  a  small  donut-shaped  SE,  one  sees  that  this  operation  has  the 
effect  of  clipping  the  input  i  such  that  no  pixel  has  a  value  greater  than  the  maximum 
value  of  its  four  nearest  neighbors. 

To  fill  in  the  dropouts,  we  replace  each  of  them  by  the  corresponding  value  in  the 
closing  of  t  by  b  (p.  37-4),  i.e., 

tb  t  €  {0,1, 2, 3} 

u  =  (4.1) 

t  otherwise. 

This  has  the  effect  of  “patching  up”  the  holes  created  by  the  dropouts  (p.  37-5).  Note 
that  the  region  of  support  of  b  will  determine  the  size  of  the  patch.  In  this  case  the 
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Noise  removal. 


region  of  support  of  the  SE  is  4  x  2  pixels  so  any  region  of  dropouts  larger  than  this 
will  not  be  patched.  With  our  data,  this  SE  was  sufficient  in  almost  all  cases. 

Lastly,  the  remaining  outliers  which  appear  as  dark  spots  must  be  removed.  This  is 
accomplished  in  a  manner  completely  analogous  to  that  for  removing  bright  spots.  We 
take  the  pointwise  maximum  of  u  and  its  erosion  uQa  (p.  37-6),  i.e.,  u  =  mar(u,(u© 
a))  (p.  37-7).  If  one  looks  at  the  formula  for  erosion  (Eq.  2.16),  the  value  of  v  at  a 
point  x,  v(x),  can  be  expanded  to 

v(x)  =  [mai(u,  (u  0  a))](i)  =  [mai(u,  (u  0  a))](i) 

=  max(u(x),min(u(x  —  y)  +  0))  y  €  Ros(a). 

By  noting  that  Ros(a)  is  a  small  donut-shaped  SE,  one  sees  that  this  operation  has 
the  effect  of  clipping  the  input  u  such  that  no  pixel  has  a  value  less  than  the  minimum 
value  of  its  four  nearest  neighbors. 

About  the  SEs 

One  may  wonder  how  the  SEs  for  this  application  were  chosen.  For  SE  a,  the  region 
of  support  was  chosen  to  cover  the  four  nearest  neighbors.  The  constant  value  of  the 
function  a  was  chosen  to  be  zero;  changing  the  constant  value  would  have  had  the 
following  effect.  If  the  value  corresponding  to  a  was  6,  the  values  of  the  function  i  ©  a 
would  have  increased  by  6;  the  values  of  the  function  i  Qa  would  have  decreased  by  6. 
Thus,  rather  than  clipping  the  outliers  to  within  the  max  and  min  of  the  four  nearest 
neighbors,  they  would  have  been  clipped  to  within  max  +  6  and  min  —  6  of  the  four 
nearest  neighbors. 

The  shape  of  SE  b  was  chosen  to  “patch”  up  a  region  of  the  image  of  the  same 
size  or  smaller.  Though  the  value  of  the  the  function  b  was  zero,  it  could  have  had 
any  other  constant  value  and  the  results  would  have  been  the  same. 
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4.3.2  Feature  Extraction 


In  most  of  the  real  range  images,  good  features  to  use  for  recognition  are  characteris¬ 
tic  object  appendages  such  as  antennas  and  guns.  Their  extraction  is  carried  out  on 
binary  image  silhouettes  since  the  real  range  data  provides  only  coarse  3-D  informa¬ 
tion.  A  binary  silhouette  is  produced  by  thresholding  the  gray-scale  image  (p.  40-1) 
at  the  median  intensity  value  (p.  40-2).  The  proposed  procedure  for  isolating  the 
characteristic  appendages  of  the  silhouette  is: 

1.  Isolate  all  appendages  with  widths  similar  to  those  of  the  characteristic  ap¬ 
pendages. 

2.  Join  isolated  appendages  if  they  axe  within  a  given  distance  of  each  other. 

3.  Filter  out  any  appendage  which  is  too  small  to  possibly  be  a  characteristic 
appendage. 

To  perform  these  steps,  only  set  MM  and  point  operations  are  used.  The  results  and 
the  SEs  used  are  shown  on  page  40. 

To  isolate  all  appendages  we  take  the  set  difference  of  the  original  binary  image, 
say  7  (p.  40-2),  and  its  opening  I&  (p.  40-3),  i.e.,  T  =  J  —  7^  =  70  (Ia)c  (p-  40-4). 
Note  that  we  have  chosen  A  to  be  slightly  larger  than  the  appendages  we  wish  to 
extract. 

For  joining  isolated  appendages,  we  use  the  closing  U  =  TB  (p.  40-5).  All  isolated 
appendages  separated  by  gaps  less  than  the  size  of  B  (4  x  2  pixels)  will  be  joined 
together. 

The  last  step  is  to  filter  out  any  appendages  which  are  too  small  to  merit  further 
study.  This  filtering  procedure  is  a  two-step  process.  First,  an  image  containing 
only  those  appendages  (artifacts)  which  are  to  be  filtered  out  is  created.  This  is 
accomplished  by  using  the  hit-or-miss  transformation  and  Minkowski  addition.  The 
hit-or-miss  transformation  will  produce  an  image  which  is  “true”  (white)  at  every 
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location  where  a  given  pattern  T 1  is  located,  provided  T2  is  chosen  as  the  complement 
of  Tl  in  a  selected  surrounding  window  [7]  (p.  40-6).  Prom  this  result,  an  image  of 
removable  artifacts  can  be  created  by  replicating  the  pattern  Tl  at  each  “true”  value 
in  the  hit-or-miss  transformation,  i.e.,  V  =  (U  ©  (Tl,  T2))0T1  (p.  40-7).  The  desired 
result  is  the  set  difference  of  the  image  of  joined  appendages  U  and  the  removable 
artifacts  V,  i.e.,  W  =  U  —  V  =  U D  Vc  (p.  40-8).  For  our  purposes,  appendages  which 
are  as  small  as  or  smaller  than  2x1  are  filtered  out.  This  implies  two  filterings;  one 
for  2x1  pixel  appendages  and  another  for  1  x  1  pixel  appendages.  The  intermediate 
steps  for  filtering  out  the  2x1  pixel  appendages  are  shown  in  p.  40-6  through  p.  40- 
8.  The  final  result,  after  filtering  out  the  lxl  appendages,  is  shown  in  (p.  40- 
9).  It  should  also  be  noted  that  if  the  approximate  object  orientation  is  known, 
MM  techniques  could  also  be  used  to  distinguish  between  vertical  and  horizontal 
appendages.  This  is  discussed  further  in  the  next  subsection  of  this  chapter. 

About  the  SEs 

SE  A  was  chosen  large  enough  so  that  it  would  not  fit  in  the  appendage  region, 
yet  small  enough  that  it  would  fit  in  the  body  region.  SE  B  was  chosen  so  that 
it  would  connect  pixels  within  4  pixels  horizontally,  2  pixels  vertically.  Ideally,  A, 
B ,  and  (T1,T2)  should  be  symmetric.  In  terms  of  the  geometry  of  the  world,  they 
are  symmetric  but  appear  non-symmetric  because  the  horizontal  angular  sampling 
interval  is  one  half  that  of  the  vertical  angular  sampling  interval. 

4.3.3  More  Examples 

With  the  previous  feature  extraction  method,  no  knowledge  of  object  orientation 
was  assumed.  However,  under  certain  circumstances,  some  assumption  may  be  made 
about  object  orientation,  e.g.  one  can  assume  that  an  object  lies  on  a  nearly  horizontal 
surface.  In  that  case,  one  may  be  able  to  reconnect  and/or  extract  appendages  at 
characteristic  orientations.  MM  algorithms  which  incorporate  a  priori  orientation 
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information  are  presented  in  the  following  paragraphs  and  typical  results  axe  shown 
on  pages  43-44  and  pages  46-47. 

The  first  example,  on  pages  43-44,  shows  techniques  of  function  MM  and  point 
operations  applied  to  a  real  range  image.  The  goal  is  to  extract  the  object  of  interest 
from  the  original  image  and  to  reconnect  broken  appendages.  The  SEs  (or  kernels) 
which  are  used  are  labelled  1  through  7.  SEs  1  and  3  are  small  ellipsoids;  the  other 
SEs  are  rectangles  of  various  sizes.  Results  at  each  stage  in  the  processing  axe  shown 
in  photos  A-K  along  with  a  description  of  the  operations  performed.  The  following  is 
a  high-level  description  of  the  algorithm. 

First,  a  relatively  clean  range  image1  ,  A,  is  closed  by  a  small  half-sphere  slightly 
larger  that  the  widths  of  the  gun  and  antennas.  This  removes  the  appendages  (B). 
Note  that  there  is  one  location  along  the  gun  barrel  where  the  SE  “fell  into  the  gun 
valley”  thereby  leaving  a  maxk  (the  “cross”)  directly  related  to  the  shape  of  the  SE. 
Next,  the  vehicle  body  remaining  in  image  B  is  “shaved  off”  (C)  by  using  a  very  long 
SE  which  must  be  longer  than  the  body  length. 

After  pointwise  subtraction  of  C  from  B,  all  that  is  left  (D)  is  the  object  plus  some 
ground  clutter,  which  is  subsequently  removed  (E)  by  closing  by  a  small  vertical  SE 
with  length  slightly  larger  than  that  of  the  largest  clutter  height.  Image  E  therefore 
contains  the  isolated  body.  Similaxly,  pointwise  subtraction  of  B  from  A  produces  a 
noisy  image  F  of  the  isolated  broken  appendages  (gun  and  antennas). 

This  image  is  now  processed  with  two  independent  sequences  of  opening  followed 
by  closing.  In  the  first,  opening  with  the  small  vertical  SE  reconnects  the  antenna 
fragments  by  eliminating  the  peaks  between  them,  while  subsequent  closing  with  a 
slightly  longer  SE  eliminates  vertical  appendages  which  axe  too  short.  The  end  result 
(H)  is  a  pair  of  less  fragmented  antennas,  no  gun,  and  very  little  noise.  The  other 
sequence  is  very  similax,  but  uses  horizontal  SE’s:  the  result  in  J  shows  the  gun,  no 
antennas,  and  very  little  noise.  By  pointwise  addition  of  the  body  image  E,  the 

'The  cleaning  procedure  is  described  in  [69]. 
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Application  of  function  MM. 


OPENING  OF  I 


44 


Application  of  function  MM  (continued). 


antenna  image  H,  and  the  gun  image  J,  one  obtains  the  reconstituted  object  with  less 
fragmented  appendages  (K).  Image  L  is  simply  a  histogram-based  segmentation  of  K 
into  two  regions. 

For  completeness,  it  should  be  noted  that  a  very  similar  processing  sequence  has 
been  inserted  in  the  processing  chain  described  in  [69]  to  reconnect  appendages  prior  to 
edge  detection.  This  additional  processing  generally  results  in  better  range  silhouette 
extraction. 

One  can  also  use  a  similar  algorithm  using  set  MM  to  extract  the  object  and 
reconnect  broken  appendages.  The  results  of  this  processing  axe  shown  on  pages  46- 
47.  The  SEs  which  are  used  are  labelled  1  through  8.  SEs  1  and  4  axe  small  ellipses; 
the  other  SEs  axe  rectangles  of  various  sizes.  Results  at  each  stage  in  the  processing 
are  shown  in  photos  A-N  along  with  a  description  of  the  operations  performed.  The 
following  is  a  brief  description  of  the  algorithm. 

The  first  step  is  to  create  a  binary  image  from  the  original  (A)  through  a  segmen¬ 
tation  procedure.  This  procedure  gives  a  silhouette  of  the  object  with  many  holes  and 
ground  clutter  (B).  The  holes  axe  filled  by  closing  by  a  small  disk  (C).  The  appendages 
are  removed  by  the  opening,  giving  an  image  of  the  object  body  (D).  The  next  step 
is  to  detect  a  continuous  horizontal  band  larger  than  the  body,  which  corresponds  to 
ground  clutter.  To  accomplish  this,  the  image  of  the  body  (D)  is  opened  by  a  long, 
horizontal  SE.  Once  the  band  (E)  is  located,  it  is  dilated  by  a  long  vertical  SE  so 
that  the  lower  half  of  the  image  (where  the  ground  clutter  appears)  is  entirely  filled 

(F) .  This  image  of  ground  clutter  can  be  subtracted  from  the  image  of  the  body  and 
ground  clutter  (D)  to  give  an  image  of  the  body  and  some  residual  ground  clutter 

(G) .  From  this  step  on,  the  extraction  and  reconnection  of  the  gun  and  antenna  is 
entirely  analogous  to  the  procedure  outlined  for  the  gray-scale  case  and  the  end  result 
(N)  is  very  similar. 
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Application  of  set  MM. 
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Application  of  set  MM  (continued). 


Chapter  5 


Experiments  with  Synthetic  Range 
Imagery 


Although  MM  techniques  give  fairly  good  feature  extraction  results  with  the  real 
data,  they  are  limited  by  several  factors.  The  coarse  range  resolution  of  the  real  data 
does  not  provide  much  information  on  the  3-D  structure  of  the  objects  of  interest;  it 
provides  only  rough  silhouettes.  Thus,  an  object  cannot  be  easily  identified  if  its  char¬ 
acteristic  appendages  do  not  appear  on  the  silhouette.  Also,  the  angular  resolution 
is  very  coarse  compared  to  the  dimensions  of  the  objects  of  interest.  This  makes  it 
difficult  to  distinguish  between  guns  and  antennas  since  they  have  approximately  the 
same  diameter  (1  or  2  pixels).  It  is  thus  natural  to  ask  how  MM  feature  extraction 
techniques  would  work  on  data  with  higher  resolution  in  range  and  angular  dimen¬ 
sions.  To  have  complete  flexibility  in  investigating  this  question,  a  synthetic  range 
image  generator  was  developed. 

5.1  Data 

The  synthetic  range  images  are  produced  from  a  description  of  the  surface  of  an 
object  in  terms  of  triangular  facets,  thus  all  objects  in  the  images  are  polyhedra.  The 
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polyhedral  objects  under  study  are  free  floating  in  space  in  front  of  a  background 
plane  perpendicular  to  the  viewing  direction.  The  horizontal  and  vertical  angular 
sampling  intervals  are  both  one  half  the  horizontal  angular  sampling  interval  of  the 
real  data,  therefore  zooming  is  not  needed  for  display.  The  range  resolution  of  the 
synthetic  data  is  60  times  better  than  that  of  the  reed  data,  and  the  distance  to  the 
object  is  approximately  the  same  as  that  for  the  real  data. 


5.2  Experiments 

In  the  next  two  subsections,  the  synthetic  data  is  used  to  explore  the  problems  of  3-D 
feature  extraction  and  corner  extraction. 

5.2.1  Feature  Extraction 

An  example  of  a  synthetic  range  image  is  shown  in  (p.  50-1).  Note  that  none  of  the 
characteristic  appendages  (gun,  antenna)  appear  on  the  silhouette,  and  that  they  have 
distinguishable  differences  in  thickness. 

We  can  use  techniques  from  function  MM  to  extract  characteristic  appendages 
from  the  synthetic  data  .  The  proposed  procedure  is: 

1.  Find  guns. 

2.  Find  antennas. 

The  results  and  the  SEs  used  are  shown  on  page  50.  Each  SE  is  a  2-D  function  whose 
3-D  graph  is  a  half-sphere. 

To  find  the  gun,  we  subtract  the  original  image,  say  i  (p.  50-1),  from  the  closing 
i°,  i.e.,  t  =  i°  —i  =  RBa(i)  (p.  50-2).  Conceptually,  this  has  the  same  effect  as  rolling 
a  ball  around  on  the  upper  surface  of  the  3-D  graph  corresponding  to  the  range  image 
and  retaining  the  places  where  the  ball  does  not  fit.  If  the  ball  is  slightly  bigger  than 
the  diameter  of  the  gun,  the  gun  will  show  up  in  the  result.  The  next  step  is  to  remove 
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CLOSING  OF  1  BV  c,  5:  OPENING  OF  4  BY  d 

MINUS  1 


appendages  and  artifacts  which  have  a  diameter  slightly  less  than  that  of  the  gun. 
This  is  accomplished  by  the  opening  u  =  tj,  (p.  50-3). 

We  can  locate  the  antenna  in  the  same  way.  This  time  we  use  SE  c  which  has  a 
diameter  slightly  larger  than  the  antenna,  so  v  =  ic  —  i  =  RBc(i )  (p.  50-4).  Again, 
we  can  remove  any  artifacts  with  diameter  smaller  than  that  of  the  antenna  by  the 
opening  w  =  vj  (p.  50-5). 

About  the  SEs 

The  choice  of  SEs  for  feature  extraction  was  relatively  easy.  Hemispherical  functions 
were  chosen  so  that  processing  would  be  independent  of  orientation.  The  sizes  of  a 
and  b  were  chosen  according  to  the  thickness  of  the  gun  and  together  they  form  a 
“size-pass  filter,”  passing  only  appendages  smellier  than  a  but  larger  than  b.  The  SEs 
c  and  d  were  selected  as  components  of  a  smaller  “size-pass  filter”,  according  to  the 
thickness  of  the  antenna. 

5.2.2  Corner  Extraction 

With  the  previous  methods  of  feature  extraction,  it  is  assumed  that  an  object  has  some 
distinguishable  appendage.  If  this  is  not  the  case,  another  feature  which  can  assist  the 
recognition  of  man-made  objects  is  the  3-D  corner.  Corner  detection  algorithms  can 
be  readily  tested  on  images  produced  by  the  synthetic  data  generator.  This  subsection 
describes  an  algorithm  developed  to  locate  convex1  polyhedral  corners  (vertices)  in 
high-resolution  range  images.  The  results  of  the  corner  extraction  and  the  SEs  used 
are  shown  on  pages  52  and  53.  SE  a  is  a  hemispherical  function;  B  and  C  are  2-D 
sets.  The  other  SEs  are  thinning  SEs. 

The  object  selected  for  this  experiment  is  a  cube  and  the  corresponding  range 

'Convex  corners  were  chosen  for  discussion  purposes;  a  minor  change  to  the  algorithm  would  provide 
for  extraction  of  concave  corners.  The  case  where  both  convex  and  concave  corners  are  present  in  the 
same  image  is  not  addressed  here. 
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J-D  corner  extraction 


RESULTS 
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3-D  comer  ext ractioni (continued). 


image  is  shown  in  p.  52-1.  The  corners  of  any  polyhedral  object  at  a  given  orientation 
can  be  divided  into  two  categories:  exposed-facet  corners  (E-corners)  and  hidden- 
facet  corners  (H-comers).  This  is  illustrated  in  p.  52-2  for  the  selected  view  of  the 
cube.  It  should  be  noted  that  the  labelling  of  corners  varies  as  the  viewing  direction 
is  changed.  With  this  in  mind,  the  problem  solving  approach  is  as  follows: 

1.  Detect  corner  regions. 

2.  Localize  corners. 

The  first  step  is  to  find  the  approximate  location  of  comers  of  each  type.  The  first 
type  of  corner  to  be  detected  is  the  H-corner.  These  comers  appear  on  the  occluding 
boundary  of  the  object  and  can  be  detected  using  the  rolling  ball  transform.  The 
SE  used  in  the  rolling  ball  transform  is  the  hemispherical  function  a.  If  the  input 
image  is  denoted  t  (p.  52-1),  the  result  of  the  rolling  ball  transform  is  t  =  RBa(i) 
(p.  52-3).  If  the  object  of  interest  stands  out  from  its  background  (such  as  an  airplane 
against  the  sky),  the  rolling  ball  will  produce  large  peaks  near  corners  on  the  occluding 
boundary.  To  understand  this,  refer  to  Figure  5.1  which  shows  the  top  view  of  a  cube 
in  the  vicinity  of  an  H-corner.  To  compute  the  rolling  ball  transform,  one  can  either 
consider  a  ball  rolling  on  the  top  surface  of  the  corresponding  range  image  (and  a 
closing),  or  a  ball  rolling  on  the  bottom  of  the  combined  object/background  surface 
(and  an  opening).  The  second  approach  is  taken  in  Figure  5.1,  where  the  lengths  of 
the  parallel  segments  shown  on  the  graphs  at  the  bottom  correspond  to  the  values  of 
the  transform  along  two  of  its  cross-sections.  Clearly,  the  maximum  values  are  found 
near  the  corner  and  constitute  a  ridge  of  constant  values.  Assuming  that  this  ridge 
is  higher  them  peaks  produced  by  vertical  (toward  the  viewer)  object  protrusions,  the 
H-corners  are  easily  found  by  thresholding  t ,  producing  a  binary  image  with  “ones”  at 
points  where  t  >  threshold.  This  binary  image  is  then  dilated  by  SE  B  to  produce  an 
image  of  detected  H-corner  regions  (p.  52-4).  The  dilation  by  B  accounts  for  effects 
of  discretization  which  can  cause  fragmentation  of  the  corner  regions.  This  effect  is 
discussed  in  the  paragraph  below  titled,  “About  the  SEs.” 
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Figure  5.L  Peaks  in  the  rolling  ball  transform . 
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Figure  5.3.  Pixel  patterns  indicating  the  presence  of  corners. 

The  next  step  is  to  detect  E-corners.  If  one  had  a  line  drawing  of  a  polyhedron 
(with  hidden  lines  removed),  one  would  notice  that  all  exposed-facet  corners  (E- 
corners)  appear  in  the  drawing  at  the  intersection  of  three  or  more  edges  (p.  52-2). 
Thus,  with  a  line  drawing  of  a  polyhedron,  one  can  detect  El-corners  (and  some  H- 
corners)  by  detecting  these  edge  intersections.  This  is  the  approach  taken  to  detect 
E- corners. 

The  line  drawing  is  obtained  in  the  following  way:  first,  the  rolling  ball  transform 
of  the  input  t,  RBa(i),  is  thresholded  at  zero.  The  “ones”  in  this  binary  image  indicate 
pixels  where  RBa(i )  >  0.  This  binary  image  T  is  then  dilated  by  B  to  correct  for  any 
artifacts  caused  by  digitization,  giving  U  =  T  ©  B  (p.  52-5).  U  is  then  thinned  by  the 
infinite  sequence  of  90-degree  rotations  of  the  hit-or-miss  SEs  L,  E,  and  LE2  (Fig.  5.2), 
denoted  {L,  £,  LE}^.  This  produces  the  thin  line  drawing  V  =  U  O  {E,  E,  LE} « 
(p.  52-6). 

The  hit-or-miss  transformation  can  be  used  to  detect  the  pixel  patterns  in  the  thin¬ 
ning,  V ,  which  indicate  the  presence  of  edge  intersections.  Examples  of  these  patterns 
are  shown  in  Figure  5.3.  Each  of  these  patterns  has  one  thing  in  common — the  pixel 
neighborhood,  when  read  in  a  clockwise  or  counterclockwise  direction,  contains  the 

2This  notation  is  motivated  by  the  Golay  alphabet  [62],  though  it  is  not  identical. 
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Figure  5.4.  Corner-detecting  SEs. 

sequence  0,1, 0,1,0  (black  =1).  In  most  cases,  this  pattern  indicates  the  intersection 
of  three  or  more  edges,  however  there  is  one  pixel  configuration3  which  matches  this 
pattern  yet  does  not  correspond  to  the  intersection  of  three  or  more  edges  (Fig.  5.4, 
NC-0).  Thus,  the  hit-or-miss  transformation  with  the  four  90-degree  rotations  of  C-0 
(Fig.  5.4),  denoted  {C-0,  C-90,  C-180,  C-270),  gives  edge-intersection  locations  and 
also  some  “false  alarms”  (occurences  of  the  four  rotations  of  NC-0).  To  eliminate 
these  “false  alarms,”  one  can  subtract  out  occurences  of  the  four  90-degree  rotations 
of  NC-0,  denoted  {NC-0,  NC-90,  NC-180,  NC-270}.  The  rest  of  the  edge  intersections 
are  found  using  the  hit-or-miss  transformation  with  the  four  90-degree  rotations  of 
C-45  (Fig.  5.4),  denoted  {C-45,  C-135,  C-225,  C-315).  Thus,  the  edge  intersections, 
W ,  are  extracted  from  the  thinning,  V,  according  to  the  following: 

W  =  (( V  ©  {C-0,  . . .  C-270})/(V  ©  {NC-0,  . . .  NC-270}))  U  (V  ©  {C-45,. . .  C-315}) 

(5.1) 

The  binary  image  W  (p.  52-7)4  is  “true”  near  the  intersections  of  three  or  more 

3One  can  find  other  neighborhoods  which  obey  this  condition  and  are  not  edge  intersections.  Under 
more  careful  observation,  however,  one  will  find  that  these  configurations  are  not  possible  after  thinning 
with  {L,  E)  LE}oo  is  performed. 

4The  large,  thick  “donuts”  in  p.  52-7  are  intended  only  to  draw  attention  to  the  single-pixel  E-corner 
locations.  They  are  not  part  of  the  image. 
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Figure  5.5.  Several  responses  for  one  corner . 

edges,  yet  the  locations  of  these  “true”  points  generally  does  not  coincide  with  the 
exact  locations  of  corners5.  Also,  the  intersection  of  more  than  three  edges  at  a 
corner  will,  in  general,  produce  several  “true”  values  corresponding  to  the  same  corner 
(Fig.  5.5).  To  solve  these  problems,  the  image  W  cam  be  dilated  by  a  disk  C .  This 
transforms  the  image  W  from  points  to  regions  corresponding  to  the  presence  of 
corners  (p.  52-8). 

It  is  now  a  simple  matter  to  combine  the  image  of  the  H-corner  regions  (p.  52-4) 
and  the  image  of  edge-intersection  regions6 7  (p.  52-8)  to  get  an  image  of  all  corner 
regions  (p.  52-9). 

After  the  corner  regions  are  labelled^,  localization  is  accomplished  by  selecting  the 
point  within  each  region  where  the  curvature  is  greatest.  The  curvature  is  measured 
as  follows  (see  Figure  5.6  for  an  example  in  1-D): 

1.  A  point  of  the  corner  region  is  selected. 

5This  is  due  to  asymmetries  in  the  thinning  process. 

6Note:  The  E-corner  regions  constitute  a  subset  of  the  edge-intersection  regions. 

7Although  the  MM  algorithm  for  region  labelling  (Section  3.2)  could  have  been  used,  running  it  on 

a  sequential,  general  purpose  computer  is  very  slow.  Thus,  a  simple  labeller  described  by  Winston  and 
Horn  ([70]  p.  131)  is  used  for  this  step. 
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Figure  5.6 .  Corner  measurement . 


2.  The  corresponding  point  fc  in  the  range  image  /  is  dilated  by  a  hemisphere  g 
and  also  by  an  inverted  hemisphere  —g. 

3.  The  curvature  is  computed  according  to  the  following  formula: 

£((/c©3)-/)>o ((/c  E((/c©-i)-/)>o((/c  ©  -g)  ~  f)  w  A_ 

- — -  X  47T 

2  x  Zg 

This  gives  an  estimate  of  the  number  of  steradians  taken  up  by  the  surface  /  at 
each  point  in  the  corner  region.  The  point  which  takes  up  the  minimum  number 
of  steradians  is  the  located  corner. 


Results 

The  results  of  the  corner  extraction  axe  shown  on  page  53.  The  originals  are  fill 
different  rotations  of  the  same  cube.  The  detected  corner  regions  show  up  on  the 
second  row  of  photographs,  and  the  localized  corners  show  up  on  the  last  row.  The 
E-corners  are  labelled  by  the  number  of  steradians  that  they  take  up,  as  measured 
from  the  original  range  images.  Ideally,  the  measurement  should  be  precisely  equal 
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to  7r/2  independent  of  orientation.  It  should  be  noted  that  this  ideal  result  may  be 
hard  to  obtain  when  distortions  due  to  perspective  are  important. 


About  the  SEs 

The  hemispherical  SE  a  was  selected  for  the  rolling  ball  transform,  RBa(i),  for  several 
reasons: 

•  It  is  rotationally  symmetric. 

•  Its  radius  of  seven  pixels  is  large  enough  to  not  fit  in  most  polyhedral  edges. 

If  the  radius  is  too  small,  the  rolling  ball  transform  will  not  pick  up  edges.  This  is  due 
to  digitization  effects  which  allow  a  digital  ball  to  “fit”  in  regions  beneath  a  digital 
surface  where  a  continuous  ball  would  not  fit  beneath  the  corresponding  continuous 
surface  (see  Fig.  5.7). 

The  smoothing  SE,  was  chosen  to  reduce  fragmentation  artifacts  produced  by 
digitization.  An  example  of  such  artifacts  appearing  in  the  digital  case,  but  not  in 
the  continuous  case,  is  shown  in  Figure  5.8.  It  is  clear  that  different  choices  for  the 
SE  a  will  create  different  fragmentation  artifacts.  Thus,  the  choice  of  B  is  directly 
related  to  that  of  a. 

The  SE  C,  used  to  dilate  the  edge-intersection  locations,  IF,  of  the  thinning,  F, 
was  selected  because: 
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Figure  5.8 .  Digital  artifacts. 


Figure  5.9 .  Corners  overlapped  by  the  corner-measuring  SE 

•  It  is  rotationally  symmetric. 

•  It  is  assumed  that  the  edge-intersection  locations  in  the  thinning,  V,  are  not 
more  than  the  radius  of  a  (seven  pixels)  away  from  the  exact  corner  locations. 
This  is  the  case  in  general. 

The  hemispherical  comer-measuring  SE,  a,  was  selected  because  its  radius  was 
large  enough  to  provide  a  good  estimate  of  the  curvature,  yet  small  enough  to  avoid 
overlapping  several  comers  at  one  time.  An  example  of  this  problem  is  shown  in 
Figure  5.9. 
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Limitations 

Besides  the  limitations  which  arise  because  of  digitization  problems,  it  should  be 
pointed  out  that  the  corner  extraction  algorithm  will  clearly  not  work  (in  its  present 
state)  for  a  general  polyhedron.  If  a  polyhedron  is  not  convex,  edge  intersections 
may  not  correspond  to  corners  (Fig.  5.10).  This  would  lead  to  erroneous  results  since 
the  algorithm  relies  on  the  assumption  that  edge  intersections  correspond  to  either 
El-corners  or  H-corners.  To  solve  this  problem,  one  would  have  to  select  another 
approach. 
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Conclusion 


MM  provides  good  techniques  for  geometric  signed  analysis.  This  is  particularly  im¬ 
portant  for  range  imagery  where  there  is  a  direct  correspondence  between  the  data 
and  the  geometry  of  the  world.  Once  an  image  feature  has  been  specified  geomet¬ 
rically,  it  can  be  extracted  using  MM.  This  has  been  demonstrated  particularly  well 
with  the  appendage  and  corner  extraction  algorithms  of  Chapters  5. 

In  Chapter  4,  function  MM  proved  useful  for  removing  noise  from  real  range 
images  and,  because  of  the  limited  resolution  of  the  real  data,  both  function  and  set 
MM  techniques  proved  useful  for  feature  extraction.  It  was  also  shown  that  the  success 
of  morphological  techniques  is  directly  related  to  the  degree  of  a  priori  knowledge  of 
the  signed  feature  to  be  extracted.  Without  knowledge  of  the  approximate  orientation 
of  an  object,  one  is  restricted  to  using  rotationally  symmetric  SEs. 

In  Chapter  5,  function  MM  proved  useful  for  3-D  appendage  and  corner  feature 
extraction  from  high  resolution  synthetic  images.  Even  in  this  “ideal”  case,  however, 
digitization  problems  affected  the  performance  of  MM  techniques.  As  was  noted, 
the  geometry  of  an  object  in  the  real  world  can  be  quite  different  from  that  in  the 
corresponding  digital  range  image.  This  problem  is  especially  significant  when  a 
feature  of  interest  corresponds  to  just  a  few  pixels. 

In  spite  of  its  limitations,  MM  has  high  potential  for  machine  vision  applications. 
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The  basic  operations  of  MM  satisfy  the  criteria  of  being  general  and  parallel.  As 
we  have  seen  in  Chapters  3,  4,  and  5,  algorithms  for  texture  analysis,  image  coding, 
noise  removal,  and  feature  extraction  can  all  be  described  within  the  formalism  of 
MM.  This  fact,  along  with  solid  theoretical  foundations  and  nice  implementation 
properties,  make  the  basic  operations  of  MM  highly  promising  candidates  for  hardware 
implementation.  At  this  point  in  time,  architectures  such  as  the  Cytocomputer  [50] 
and  the  CLIP  [53]  can  support  the  basic  MM  operations.  The  MM  algorithms  for  noise 
removal  and  2-D  feature  extraction  have  also  been  implemented  by  the  authors  on 
a  wafer-scale,  parallel  image  processor  simulator.  For  machines  that  can  access  only 
a  local  pixel  window,  the  optimal  decomposition  of  SEs  becomes  an  important  task 
in  MM  algorithm  development.  Zhuang  and  Haralick  [46]  have  recently  solved  this 
problem  for  set  MM  SEs,  however,  the  decomposition  of  function  MM  SEs  remains 
an  open  research  problem. 

In  conclusion,  the  operations  of  MM  satisfy  the  criteria  set  forth  for  desirable  ma¬ 
chine  vision  operators:  they  are  general  and  can  be  implemented  in  parallel.  They  are 
designed  for  geometric  signal  analysis  and  herein  lies  both  the  strength  and  weakness 
of  MM-based  approaches  to  vision.  Although  the  techniques  of  MM  alone  will  not 
enable  a  computer  to  see,  they  go  a  long  way  toward  this  goal. 
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This  paper  presents  some  of  Maragos’  preliminary  work  on  image  coding  using 
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tessalation  is  an  aggregate  of  non-overlapping  cells  which  cover  an  image). 
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[33]  Rodenacker,  K.;  Gais,  P.;  Jutting,  U.;  Burger,  G.,  “Mathematical  Mor¬ 
phology  in  Grey-Images,”  Signal  Processing  II:  Theories  and  Applications, 
Proc.  EUSIPC0-8S,  Erlangen,  Germany,  pp.  131-4,  Publ.  North-Holland,  Sept., 
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MM  is  applied  to  the  analysis  of  human  cell  images.  The  rice  field  transforma¬ 
tion,  which  can  provide  a  reasonably  tractable  description  of  the  topology  of  an 
image,  is  introduced. 

[34]  Serra,  J.,  Image  Analysis  and  Mathematical  Morphology,  Acad.  Press,  1982. 
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under  translation,  compatibility  with  magnification,  local  knowledge,  and  semi¬ 
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ments.  These  principles  were  first  used  in  the  early  60s  by  the  english  “Quan- 
timets”  and  then  from  the  late  60s  by  the  french- german  ” Texture  Analysers”. 
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Texture  analysers  are  used  (as  of  1978)  as  follows:  50%  for  material  science,  40% 
for  biology,  and  10%  for  macroscopic  imagery  (e.g.,  aerial  photographs).  The 
wide  range  of  applications  is  illustrated  through  two  extreme  examples,  i.e., 
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MM  is  used  to  decompose  binary  image  shapes  into  primitives.  These  primitives 
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concise  and  accurate  descriptions  of  the  basic  MM  operations,  along  with  an 
intuitive  introduction  to  some  of  the  more  advanced  theory.  The  mathematical 
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